-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[Enchancement](query-context) add coordinator-context to extend lifetime about rf handle #58189
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
3 similar comments
|
run buildall |
|
run buildall |
|
run buildall |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
38eddb1 to
cb4fc02
Compare
|
run buildall |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
1 similar comment
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
|
run buildall |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
1 similar comment
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
What problem does this PR solve?
This pull request introduces a new
CoordinatorContextclass to better manage query coordination and runtime states across BE (Backend) nodes, separating responsibilities fromQueryContext. The changes refactor how runtime filter merge controllers and query coordination are handled, improve resource cleanup, and enhance debugging capabilities. The most important changes are grouped below.Introduction of CoordinatorContext
CoordinatorContextclass (coordinator_context.h,coordinator_context.cpp) to manage query coordination and runtime states beyond the lifecycle ofQueryContext. This class holds references to runtime filter merge controllers and resource contexts, and provides debug and timeout management functions. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-789e0272245b8bd410068d5444860ea98a102ff4ce756f48217d7df53ebd4abfR1-R114),[[2]](https://github.com/apache/doris/pull/58189/files#diff-6839f3ffba20c6a2e7c6727e45c07be21838302c33e196007e8dde8d29b0dd75R1-R94))Refactoring Query and Coordinator Context Usage
FragmentMgrto create and manageCoordinatorContextinstances per query, storing them in_coordinator_context_map, and linking them to their respectiveQueryContextobjects. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R731-R734),[[2]](https://github.com/apache/doris/pull/58189/files#diff-9f558888989dd08ae617466466716e106c8480524f9f1d86d7b0b650115058a1R213-R214))CoordinatorContextinstead ofQueryContext, ensuring proper lifecycle and avoiding circular references. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L758-R763),[[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L1314-R1336),[[3]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L1351-R1377))Resource Cleanup and Timeout Management
CoordinatorContextobjects are erased from_coordinator_context_mapwhen a query is cancelled, and added logic to detect and log lingering contexts that have timed out. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R911),[[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R973-R986))Debugging and Monitoring Improvements
CoordinatorContextdebug information for better observability of query coordination states. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R793-R802),[[2]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L807-L813))Minor Code Cleanups
query_context.cppand updated method signatures for clarity and consistency. ([[1]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL26),[[2]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL35),[[3]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL46-L49),[[4]](https://github.com/apache/doris/pull/58189/files#diff-cde85080314c53bf67ef65f4257c3545119819c3534d4412910f1677ce217c3aL95-L112),[[5]](https://github.com/apache/doris/pull/58189/files#diff-96a1dcbd297bc02611e5cdcfb8ebcdbde45c2603f22fb3454c0da1c7ec56261aL198-R204),[[6]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2R67),[[7]](https://github.com/apache/doris/pull/58189/files#diff-3be0bcdb0c3c5fe6415556cf49c7270d4fc2e2f071c0240145ce175a32a484e2L78),[[8]](https://github.com/apache/doris/pull/58189/files#diff-9f558888989dd08ae617466466716e106c8480524f9f1d86d7b0b650115058a1R56))Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)